Accurate weather forecasting across time scales is critical for anticipating and mitigating the impacts of climate change. Recent data-driven methods based on deep learning have achieved significant success in the medium range, but struggle at longer subseasonal-to-seasonal (S2S) horizons due to error accumulation in their autoregressive approach. In this work, we propose OmniCast, a scalable and skillful probabilistic model that unifies weather forecasting across timescales. OmniCast consists of two components, a VAE model that encodes raw weather data into a continuous, lower-dimensional latent space, and a diffusion-based transformer model that generates a sequence of future latent tokens given the initial conditioning tokens. During training, we mask random future tokens and train the transformer to estimate their distribution given conditioning and visible tokens using a per-token diffusion head. During inference, the transformer generates the full sequence of future tokens by iteratively unmasking random subsets of tokens.

Does a Variational AutoEncoder (VAE) consistently encode typical samples generated from its decoder? This paper shows that the perhaps surprising answer to this question is `No'; a (nominally trained) VAE does not necessarily amortize inference for typical samples that it is capable of generating. We study the implications of this behaviour on the learned representations and also the consequences of fixing it by introducing a notion of self consistency. Our approach hinges on an alternative construction of the variational approximation distribution to the true posterior of an extended VAE model with a Markov chain alternating between the encoder and the decoder. The method can be used to train a VAE model from scratch or given an already trained VAE, it can be run as a post processing step in an entirely self supervised way without access to the original training data. Our experimental analysis reveals that encoders trained with our self-consistency approach lead to representations that are robust (insensitive) to perturbations in the input introduced by adversarial attacks. We provide experimental results on the ColorMnist and CelebA benchmark datasets that quantify the properties of the learned representations and compare the approach with a baseline that is specifically trained for the desired property.

This study investigates the stylistic differences among various Bible translations using a Variational Autoencoder (VAE) model. By embedding textual data into high-dimensional vectors, the study aims to detect and analyze stylistic variations between translations, with a specific focus on distinguishing the American Standard Version (ASV) from other translations. The results demonstrate that each translation exhibits a unique stylistic distribution, which can be effectively identified using the VAE model. These findings suggest that the VAE model is proficient in capturing and differentiating textual styles, although it is primarily optimized for distinguishing a single style. The study highlights the model's potential for broader applications in AI-based text generation and stylistic analysis, while also acknowledging the need for further model refinement to address the complexity of multi-dimensional stylistic relationships. Future research could extend this methodology to other text domains, offering deeper insights into the stylistic features embedded within various types of textual data.

Abstract-- This paper presents a teleoperation system for controlling a redundant degree-of-freedom (DOF) robot manipulator using human arm gestures. We propose a GRU-based Variational Autoencoder (VAE) to learn a latent representation of the manipulator's configuration space, capturing its complex joint kinematics. A fully-connected neural network maps human arm configurations into this latent space, allowing the system to mimic and generate corresponding manipulator trajectories in real-time through the VAE decoder. Arrow shows the mapping relationship between the manipulator's For example, an operator can use as agriculture, healthcare medicine, warehousing, and manufacturing. Another have proliferated, such as for image generation and natural approach instead uses an external RGB and RGBD (depth) language generation, for example ChatGPT, Midjourney, camera to estimate the operator's 6-DOF hand pose [4], [5] and Dall-E, to name a few.

This naturally brings up the question of whether careful tuning of the scaling coefficient for the likelihood function of each dimension could ease the aforementioned optimization difficulties. The "VAE-adaptive" baseline seems to be a data-dependent attempt at this, but I'm not convinced that a single minibatch is sufficient for computing the coefficients for each data type (as described in Appendix C.1.2). In particular, it'd be interesting to see if VAEM would outperform a (possibly hierarchical) VAE with more carefully tuned scaling factors for each dimension to rule out the possibility that the poor performance of vanilla VAE baselines is simply due to hyperparameter tuning.

Does a Variational AutoEncoder (VAE) consistently encode typical samples generated from its decoder? This paper shows that the perhaps surprising answer to this question is No'; a (nominally trained) VAE does not necessarily amortize inference for typical samples that it is capable of generating. We study the implications of this behaviour on the learned representations and also the consequences of fixing it by introducing a notion of self consistency. Our approach hinges on an alternative construction of the variational approximation distribution to the true posterior of an extended VAE model with a Markov chain alternating between the encoder and the decoder. The method can be used to train a VAE model from scratch or given an already trained VAE, it can be run as a post processing step in an entirely self supervised way without access to the original training data.

This paper proposes an algorithm for learning goal-conditioned RL policy, in which a goal is defined as a single image. The authors propose to encode a state (an image) to a vector in latent space using variational autoencoder, and define reward functions inside the latent space. The paper shows that such reward function outperforms baseline such as pixel based reward functions. The authors then proposed latent goal relabeling, which generates new goals and rewards given an exist tuple (s, a, s'). Finally, the authors propose goal imagination, which samples goals from latent space during training, essentially allowing training without specifying a particular goal.